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Abstract 

In this paper, we estabhsh uniform-in-bandwidth limit laws of the logarithm for 
nonparametric Inverse Probability of Censoring Weighted (I.P.C.W.) estimators of 
the multivariate regression function under random censorship. A similar result 
is deduced for estimators of the conditional distribution function. The uniform-in- 
bandwidth consistency for estimators of the conditional density and the conditional 
hazard rate functions are also derived from our main result. Moreover, the logarithm 
laws we establish are shown to yield almost sure simultaneous asymptotic confidence 
bands for the functions we consider. Examples of confidence bands obtained from 
simulated data are displayed. 

Key words : censored regression, kernel estimates, laws of the logarithm, inverse 
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1 Introduction-Motivations 

Nonparametric estimators of functionals of the conditional law (such as the regression 
function or the conditional distribution function) are known to provide a suitable and 
efficient means to catch the possibly complex relation between a given variable of inter- 
est and some explanatory covariates. Because of this obvious practical interest, many 
authors have studied the (asymptotic) properties of such estimators (see, e.g., [3(, 19|, 



1811 ) ■ Fewer works deal with the special case where the variable of interest is censored 
JJ], [s^])- Yet, this situation arises in many statistical applications, as medical re- 



search, reliability, ... and it is therefore of paramount importance to build and study 
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estimators adapted to the censored setting. When the variable of interest is subject to 
right-censoring, transformations of the observed data are generally needed to derive in- 
ference on the underlying (conditional) distribution (see [5i]). Estimates based on these 
transformations are usually referred to as synthetic data estimates in the literature. In 
the case of the regression function estimation, Fan and Gijbels fl3 ] especially proposed 
a transformation leading to a local version of the Buckley-James estimator. In this 
paper, we make use of an alternative transformation which leads to Inverse Probability 
of Censoring Weighted [I .P.C.W.] estimators. I.P.C.W. type estimators have recently 
gained popularity in the censored literature. To our mind, they basically present two 
particularly appealing properties. First, and as it will be especially made clear in the 
proofs of our forthcoming results (see also 0], 0]) 21 1 and the references therein). 



their asymptotic behavior can be easily deduced from that of analogous estimators in 
the uncensored case. Second, their computation is straightforward. In that sense, they 
are appealing for both theoretic and applied statistics purposes. It is however note- 
worthy that methodology we propose here for I.P.C.W -type estimates shall apply with 
minor modifications to cope with other synthetic data estimates. 

The present paper is organized as follows. First, we introduce the main notations 
and hypotheses needed for our task. Then, following the methodology developed in 



the uncensored case by Einmahl and Mason we establish a uniform-in-bandwidth 
law of the logarithm for a nonparametric I.P.C.W. estimator of the regression function 
(see Theorem 13.11 below). This result corresponds to the almost sure and uniform-in- 
bandwidth version of Theorem 3.1 in [2?]. At this point we shall stress the reader 



attention on the fact that, as was first shown by Deheuvels and Mason [9| (see also [131 ], 



10( 1 and the relevant references therein), such uniform-in-bandwidth limit laws turn 
out to be of particular interest in practice because they allow for establishing uniform 
consistency of data-driven (and then random) bandwidth estimators. In Section H] we 
derive a similar law of the logarithm for an estimator of the conditional distribution 
function and we establish the uniform-in-bandwidth consistency for estimators of the 
conditional density and the conditional hazard rate functions. As was especially pointed 
out by Deheuvels and Mason [9] in the uncensored case, limit laws of the logarithm 
provide themselves useful in the construction of simultaneous confidence bands for the 
true considered function. Such confidence bands, obtained from simulated data, are 
displayed in Section HT2l Finally, Section [5] is devoted to the proofs of our results. 



2 Notations and hypotheses 

Consider a triple (y, C, X) of random variables defined in H x H x H*^, d > 1. Here 
Y is the variable of interest, C a censoring variable and X = (Xi, X^) a vector 
of concomitant variables. Throughout, we work with a sample {(1^, Cj, Xj)i<j<„} of 
independent and identically distributed replica of {Y, C, X) , n > 1 . Actually, in the 
right censorship model, the pairs (Yi,Ci), 1 < i < n are not directly observed and the 
corresponding information is given by Zi := nim{Yi,Ci} and Si := I|y.<c.}, 1 < i < n, 
with TLe standing for the indicator function of E. Accordingly, the observed sample is 
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Vn = {{Zi,6i,Xi),i = 1, . . . ,n}. 

In the sequel, we impose the fohowing assumptions upon the distribution of {X,Y). 
Denote by I a given compact of M!^ with non empty interior and set, for any 7 > 0, 

P = {x : inf \x - u|f{d < 7}, 

with I • \^d standing for the usual euclidian norm on M!^. We will assume that, for a 
given a > 0, {X,Y) [resp. X] has a density function fx,Y [resp. fx] with respect to 
the Lebesgue measure on /" x H [resp. P]. We will also assume that the assumptions 
(F.1-2) below hold. For -00 < t < 00, set F{t) = F{Y < t), G{t) = P(C < t) 
and H{t) = P(Z < t), the right continuous distribution functions of Y, C and Z 
respectively. For any right continuous distribution function L defined on H, further 
denote by Ti = sup{t E H : L{t) < 1} the upper point of the corresponding distribution. 

(F.l) For all x £ lim /x,y(^c',y) = fxxi^^u) ^'^^ almost every y < Th- 

(-F.2) fx is continuous and strictly positive on I". 

Now consider a pointwise measurable class J- (see p. 110 in (25| ) of real measurable 
functions defined on M. Throughout, will be assumed to form a VC subgraph class 
(see §2.6.2 in Q)- 

In this paper, we will mostly focus on the regression function of ip{Y) evaluated at 
X = X, for £ and a; G given by 

m^(x) =E(^(y) I X = x). (2.1) 

To estimate when Y is right-censored, the key idea of I.P.C.W. estimators is as 
follows. Introduce the real valued function defined on by 

^ / N i{j/<c}V'(y A c) 

^^(^'^^= l -G(yAc) • 

Assuming the function G to be known, first note that ^^{Yi, Ci) = 6iip{Zi)/{l — G{Zi)) 
is observed for every 1 < i < n. Moreover, under the assumption (T) below, 

(T) C and {Y,X) are independent; 

we have 

m$^(x) := E($^(y,C)|X = a:) 



1 - G{Z) 
G{YY 

m^(x). (2.3) 
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Therefore, any estimate for m^^ , which can be built on fully observed data, turns out to 
be an estimate for too. Thanks to this property, most statistical procedures known 
to provide estimates of the regression function in the uncensored case can be naturally 
extended to the censored case. For instance, kernel-type estimates are particularly easy 
to construct. Let K he a kernel function defined on H'^, that is a measurable function 
such that f-^d K{x)dx = 1, and set, for x G /, /i > 0, 1 < i < n, 

^.,,,(.) := k[^^) I ±k{^-^) . (2.4) 

i=i 

Unless otherwise specified, we will let /i > vary in such a way that h!^ < h < /i", 
where {h'n}n>i and {h'^}n>i are two sequences of positive constants such that < /i^ < 
h'^ < oo and, for either choice of /i„ = h'^ or hn = h'^, the conditions (i/. 1-2-3) below 
are fulfilled by {hn}n>i- 

(H.l) /i„ i 0, < /i„ < 1, and nh^ ] oo; 

{H.2) nhf^/ log n ^ oo as n ^ oo; 

{H.3) log(l//i„)/loglogn ^ oo as n ^ oo. 

At this point, we shall make the reader note that, under (iJ. 1-2-3) and {F.2), the 
denominator involved in the expression of the functions Wn.h,i is almost surely strictly 
positive on / for n large enough, for every \ <i <n and all h G [/i^ , /i"]. 

In view of (|2.2p . (j2.3|) and (j2.4|) . whenever G is known, a kernel estimator of m^(x) is 
given by 

m^,n,h{^) ■= Z^'^n,h,ii^) ^ _ G{Z y ^ ' 

In practice however, the function G is generally unknown and then has to be estimated. 
We will denote by G* the Kaplan- Meier estimator of the function G [2^. Namely, 
adopting the conventions = 1 and O'' = 1 and setting Nn{x) = Y17=i ^{Zi>x}i "^e 
have 

Gi{u) = 1 - n (^^Tnk^) ' ^ ^ ^- (^-^^ 

i:Zi<u 



Given these notations, the following estimator of m^{x) can be proposed (see 2j] for 
instance) , 

Adopting the convention 0/0 = 0, this quantity is properly defined since G'^{Zi) = 1 if 
and only if Zj = and = 0, where is the fc-th ordered statistic associated 
to the sample {Zi, Z„) for k = 1, ...,n and 6(^1^^ is the 6j corresponding to = Zj. 
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As mentioned in pi|, functionals of the (conditional) law can generally not be estimated 
on the complete support when the variable of interest is right-censored. Accordingly, in 
order to establish our results, we will work under the assumption (A) that will be said 
to hold if either (A)(i) or (A)(ii) below holds. 

(A)(i) There exists a tq < Th such that, for all ^ G .F, = on (tq, cxd). 

(A)(ii) (a) Foragiven0<p<l/2, /(f^(l-F)-P/(i-P)dG<oo; 
(6) Tf < Tg and {Y, C) e 1R+ x 1R+; 
(c) n^^^-^/i^^'^l log/i"| ^ oo, as n — > oo. 

Remark 2.1. The assumption (A)(ii) will be needed in our proofs when considering the 
estimation of the "classical" regression function, which corresponds to the choice ip{y) = 
y. On the other hand, rates of convergence for estimators of functionals such as the 
conditional distribution function W{Y < t|X) can be obtained under weaker conditions, 
when restricting ourselves to t £ [— oo,ro] with tq < Th- Compare the conditions in 
Theorems \3.1\ and Corollary \4-l\ below. 

Besides the above assumptions, we will impose the following additional hypotheses to 
establish our main results. 



(F.3) The variable C has a Lebesgue density function fc on {—oo,Th]- 

(FA) The class of functions J- is bounded (in the sense that has a measurable and 

uniformly bounded envelope function T(y) > sup^^J^^p{y), y < Th)- 
{F.5) The class of functions := {m^/ fx,ip G ^} is relatively compact with 

respect to the sup-norm topology on 

It is noteworthy that (F.4-5) are automatically fulfilled for the particular choice J- = 
{I(_oo,t]!^ < Th} (see pp. 6-7 in This property will enable us to easily describe 

uniform consistency for estimators of the conditional distribution function F{t; •) := 
P(y < t\X = •) over t E (-oo, rf/)(see Corollary O in Section g]). 

Turning our attention to the kernel K, set /C := {K{X{- — y)),y G IR*^, A > 0} and 
denote by AA(e, /C) the uniform covering number of the class /C for e > 0, and the class 
of norms {-L2(P)}, with P varying in the set of all probability measures on (for 
more details, see, e.g., pp. 83-84 in pSi]). Further set |s| = maxi<j<^ \ sj\ for all s G M"^. 
We will work under the following assumptions (ii'. 1-2-3). 



(K.l) 



{K.2) 



iK.3) 



(^) 
(ii 

{ii 

{ii 



lim 



.o/r^ {K{x)-K{x + u)fdx = 0. 



limA^i J^, {K{Xx) - K{x)ydx = 0. 

For some < k < oo K{s) = 0, for |s| > |. 

For some constant < Ck < oo, sup^^gj^d |i^(a;)| < Ck- 

3 C > 0,^; > 0,V < e < l,AA(e,/C) < Ce'". 

fC is pointwise measurable. 
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Remark 2.2. (i) It is easily checked that the hypotheses (K.l) and {K.3) are especially 
fulfilled if the kernel K can he written K{x) = 4>{P{x)), where P is a polynomial in d 
real variables and (j) a real function with bounded variations. 

[ii) A class fC fulfilling the assumption {K.2>){i) is said to admit a polynomial uniform 
covering number. 

In what follows, we will make use of an auxiliary function {Q{x) : x G /}, assumed to 
be continuous and positive on I. Moreover, we will assume that the quantity Qn{x) is 
a consistent estimator of @{x), in the sense that, with probability one, 



(9.1) lim sup 



e(^) 



1 



0. 



Following the ideas of Einmahl and Mason [1^] and Deheuvels and Mason [9|], we will 
study the uniform convergence to of fh^ ^ ^ centered by the following centering factor. 

Em^;„(a;;/i)=E{V'(m(^^)}/E{A'(^^)}. 

Remark 2.3. To motivate this choice, we stress the reader attention on the fact that 
the remaining bias type term, that is Wjm^-n{x; h) — m^(x), can be neglected under some 
general regularity assumptions (see Section 2 in [5] and Theorem \4.1\ below) 

Further introduce for x £ I" and tp £ J-', 

CT^ (x) = m\Y)/[l - G{Y)] I X = x} - m2 (x). (2.8) 

Remark 2.4. Note that, under (F. 1-2-4), the function (as well as the function 
m^) introduced above is continuous on I" (see, e.g.. Section A. 3 in f^] for a complete 
demonstration of this result). It is also noteworthy that, in the uncensored case, i.e. 
when G{y) = for all y < Tp, (T^(x) is the conditional variance of ipiY) given X = x. 

3 Main result 

We have now all the ingredients to state our main result, captured in Theorem 13.11 
below. 

Theorem 3.1. Let {/i^,}n>i CLiT'd {/i^}n>i be two sequences of positive constants fulfilling 
the hypotheses (H.1-2-3) with < /i^ < /i" < 1. Under the assumptions (A), (I), (F.l- 
2-3-4-5), {K.1-2-3), and (9.1), we have with probability one, 

VrJ? ± Qn{x)lin';p ,^ f^{x) - Em^;„(ic; /i) | 

lim sup sup sup ■ 



1/2 



K [u)du sup — -— — > . (3.1) 
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It is noteworthy that in the uncensored case, i.e. when G{y) = for all y < Tp, 
the conditions (A)(m) (and then the condition (A)) , (X) and (-F.3) are automatically 
fulfilled, in such a way that Theorem 13.11 reduces in this case to a complement of 
Corollary 1 in [l^. Corollary 3.3 in [9] or Theorem 2 in [l^ . 

The proof of Theorem 13.11 is postponed to Section [5l In the following Section [H we 
present some direct consequences of Theorem 13.11 



4 Corollaries- Applications 
4.1 Corollaries 

In this Section, we show how Theorem 13.11 can be used (i)— to obtain a uniform law 
of the logarithm for an estimator of the conditional distribution function and (ii)— to 
establish the uniform-in-bandwidth consistency for some estimates of the conditional 
density and conditional hazard rate functions. The proofs of the corresponding results, 
captured in Corollaries 14.11 14.21 and 14.31 below, are postponed to Section 15.61 

Consider the following estimate of the conditional distribution function F(t;x) := 
P(y < t\X = x), along with the corresponding centering term, 

F„,,(t, X) .- ^ WnM^)-^^ and F,{t, x) .- 

Corollary 4.1. Let {h'^}n>i <ind {h'^}n>i be two sequences of positive constants ful- 
filling the hypotheses {H.1-2-3) with < /i^ < /i^ < 1. Under the assumptions (Z), 
(F.1-2-3), (i^.1-2-3), and (0.1), we have, for all tq < Th, with probability one, 

± Qnix) Snpt<ro {K,h(t-^ ^) - Ph{t; X)\ 

lim sup sup ■ 



n—roo 



= \ K\u)du sup . ^'"'-'^ . (4.1) 

( xei Jx{x) J 

To establish the next corollaries, we will work under the following additional assumption. 

{D) Th < oo and the derivatives of order one of fx and fx,Y exist and are 

bounded by a common constant on I" and x {—oo,Th) respectively. 

Moreover, denote by £ > an additional bandwidth. As for the conditional density 
f{t;x) := fx,Y{x,t)/fx{x), we consider the following estimator 



i=l 
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Corollary 4.2. Let {^'„}n>i and {^"}n>i two sequence of positive constants such 
i/iai £^ — >■ 0, £^ > £^ > — — ^^j73 — ^ . Further assume that the assumptions of Corollary 



4-l\ and (D) hold. Then we have, with probability one, 



lim sup sup sup sup I /* ;j ^(t; a;) — /(t; a;) > = 0. (4.2) 

Now, turning our attention to the conditional hazard rate function A(t; x) := f{t; x)/[l — 
F{t;x)], we introduce the following estimator 



Corollary 4.3. Under the assumptions of Corollary \4.S\ we have, with probability one, 

lim sup sup su^) sup I g{t; x) — X{t; x) > = 0) . (4-3) 

'''^°°helh'„,h'j,]£eK,e'J,]x&it<To ^ ^ 

4.2 Almost sure asymptotic simultaneous confidence bands for the 
true regression function 

4.2.1 A necessary result 

In this section, we first establish a direct consequence of Theorem 13. II which will enable 
us to build confidence bands for the theoretical regression function. The following 
notations and hypotheses will be needed. 

In this paragraph, we will impose the following regularity conditions upon the functions 
fx and fx,Y- 

{F.6) (i) fx is three times continuously differentiable on 

{"ii) fx,Y is three times continuously differentiable on x H. 

To ensure that the bias- type term may be neglected, we will work in this section with 
kernels satisfying the assumption (KA) below. 

{KA) J^,u{\..u{^K{u)du = 0, ji,...,jd>0, ji + ...+jd = 0,l,2. 
Further denote by V/ the Lebesgue measure of / and set 



log,,;,(n) := log V u[ j^^ K\t)dty^ ., 



where > 1 is a fixed constant. We refer to Remark 1.4 in for discussions about the 
introduction of these quantities along with some relevant choices for 6. 
Finally, given a sequence {/in}n>i of positive constants fulfilling the conditions {HA- 
2-3), consider a sequence of possibly data-driven bandwidths i^n(x) such that the as- 
sumption (-B.1) below holds, for two given constants < ci < C2 < cxd. 
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{B.l) cihn < inf Hn{x) < sup Hn{x) < C2hn almost surely as n ^ oo. 

Theorem 4.1. Given two constants < ^ < oo and 1/(4 + d) < 5q < 1, assume that 
Hn{x) is a bandwidth function fulfilling the assumption {B.\) with hn = An~^" . Then, 
under the assumptions (A), (I), (F.1-2-3-4-5-6), (/C.1-2-3-4) and (S-1), we have, with 
probability one, 

lim sup sup ■ 



n— >oo 



r^2(..,. e^(a;)sup^g^4(a;) |'^' 

K {t)dt sup — -- — ^ } . (4.4) 

'Rd xei Jx[x) J 

The proof of Theorem 14.11 follows from Theorem 13.11 along the same lines as the first 
part of Theorem 1.1 in l9\ is shown to be a consequence of their Corollary 3.3. We omit 
the details of this book-keeping argument. 

Remark 4.1. The assumption (B.l) above is the almost sure version of condition 
{B.l) in [§]. Theorem \3. 1\ should allow to treat even more general random bandwidths, 
especially bandwidths Hn such that h'^ < mixei Hn{x) < sup^ei -^nix) < /i" almost 
surely as n ^ oo, whenever {/i'„}n>i cLnd {/i"}n>i o-re two sequences of positive constants 
fulfilling (H.1-2-3). However, some regularity conditions have to be imposed upon the 
function Hn{-), otherwise the corresponding version of Theorem \^.l\ can not be directly 
derived from Theorem \3.1\ (the equality in ll[4-4\ ) becoming an inequality). Alternatively, 
this generalization could be derived from some functional limit law associated to Theorem 
\3.1 We especially refer the reader interested in functional limit laws of the logarithm 
to Jj/ or \23i J and the relevant references therein. 



4.2.2 Construction of the bands 

Here we give an example illustrating how Theorem 14. II mav be used to construct simul- 
taneous confidence bands for m^. Consider a random sequence of functions Ln{x) such 
that for all < e < 1, there exists almost surely an no = 'rao(e) such that, for all n > uq, 
uniformly over x £ I, 



(4.5) 



m^{x) E [m*^^rr,H„^x)(x) ± (1 + ^)Ln{x)], 
m^{x) i \m\,n,H^{x)'^^) ± (1 - ^)^n{x)\. 

Whenever (|4.5p is fulfilled for all < e < 1, we will say that the intervals 

\An(x),Bn(x)\ = [fh*^^n,H„{x)i^) - Ln{x),m*^^n,H„{x)i^) + Ln{x)] (4.6) 

provide asymptotic simultaneous confidence bands for m^{x) over x £ I. 
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It is easy to check that the quantity 



is a consistent estimate of a^{x), uniformly over tp £ and x £ I. Therefore, setting 

1/2 



nH^{x) fx;n{x,Hnix)) 



Ln{x) = < ' „ , X 



>. 1/2 




1 ' 


[ K^{t)dt 







(4.8) 



with 



1 " 

/x;n(a;; Hn{x)) = Yl ^ 



X - X,; 



nmix)^^ \H^ix)J' 
it is easily derived from Theorem 14.11 that the intervals 

[^*^,n,H^ia^)i^) - Ln{x),ml^^^H„{a^){x) + Ln{x)] (4.9) 

provide asymptotic simultaneous confidence bands for m^{x) over a; € /, in the sense 
made precise above. 

4.2.3 Illustration : a simple simulation study 

In this paragraph, the confidence bands introduced above are constructed on simulated 
data. We worked with a sample size n = 2000, and considered the case where X = 
X e IR (i.e. d = 1) was such that X ~ AA(0, 1), where AA(0, 1) stands for the gaussian 
distribution with mean and standard deviation 1. Set p{x) = 0.25 + 0.5 x cos^(a;). 
We selected -0 = I{.<o.9}) considered the model lE['i/'(^)|-'^ = x] = p{x). Under 
this model, the variable Y was simulated as follows. For each integer 1 < i < n, let 
Pi = p{xi) where Xi is the observed value of the variable Xi. Note that < pi < 1 
for every 1 < i < n. Each Yi was then generated as one ^(0.9 — pi,l + 0.9 — pi) 
variable, where U{a,b) stands for the uniform distribution on (a,b). Following this 
proceed ensured that F{Yi < 0.9\Xi = Xi) = pi = p{xi). Regarding the censoring 
variable, we generated an i.i.d. sample Ci,...,C„ such that Cj ~ UiQ^l). This choice 
yielded, a posteriori, W{5 = 1) ~ 0.2. As for K, we opted for the Epanechnikov kernel. 
Moreover, we selected Hn{x) = h = 0.15 and Hn{x) = h = 0.20. Results are presented 
in Figure [H The confidence bands appeared to be adequate, in the sense that they 
contained the true value of the regression function for every x £ [—1, 1]. The fact that 
the true function did not belong to our bands for some points was expected : it is due 
to the e term in (14.51). 



We also performed a simulation study in order to empirically evaluate the finite sample 
behavior of the distribution of the e term involved in (j4.5p . Given a sample of size n 
(simulated in the same way as above), and an estimate ^ ^(x) built on this sample, 
consider the quantity 

ei{h,n) = |m^_„ ;,(a;o) - m^(xo)| - L„.{xo), 
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(a) h=0.15 



(b) h=0.20 



Figure 1: Results of the simulation study for (a) h = 0.15 and (b) h = 0.20 : true 
additive components (solid line), their estimations (dashed line), and the associated 
confidence bands (dotted line) 

where Ln{x) is defined as in (j4.8p and 

xo = argmax {|m^^„^,,(x) - m^{x)\, x G [-1, 1]}. 

Then, by simulating 2000 samples of size n, we were able to estimate the distribution 
of ei{h,n), for various sizes n and bandwidths h. Our results are presented in Figure 
[21 As expected, the distribution of ei{h,n) is all the more concentrated around as 
n is high. Moreover, the bandwidth choice is crucial since £i{h,n) appears not to 
be centered around for some pairs {h,n). This highlights the need to build some 
procedures aiming at optimally selecting the bandwidth (in a sense to be made precise) . 
This particular problem will be addressed elsewhere. It is also noteworthy that a more 
formal description of the asymptotic behavior of the e term involved in ()4.5p could be 
achieved by studying the rate of coverage pertaining to Theorem 13.11 and Theorem 14.11 
(see [H and [3] for examples of such results) . 

5 Proofs 

To prove our results, we will first establish a general result for an estimator of the re- 
gression function in the uncensored case. Then, we will make use of the relation ()2.3p 
to treat the censored case when the function G is known. Finally, strong consistency re- 
sults for the Kaplan-Meier estimator will be employed to cope with the general censored 
case. 
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(a) h=0X5 



(b) h=0.20 



Figure 2: Boxplots of the ei{h,n) distribution for various sample sizes n and for (a) 
h = 0.15 and (b) h = 0.20. 



5.1 The uncensored case 

Let (Xi, Yi), (X2, Y2), . . ., be a sequence of independent and identically distributed 
[i.i.d.] replica of the random pair {X,Y) G R*^ x R*, d,q > 1. Consider a pointwise 
measurable class J-q of real measurable functions defined on IR^. Wc will assume that 
J^q forms a VC subgraph class and consider the conditional expectation of \1'(Y) given 
X = a;, for * e J^q, 

m^{x) = E(*(Y) \X = x). (5.1) 

In this uncensored setting, we will especially work under the assumptions (F.I-IV-V) 
below. 

{F.I) For all x e lim fx,yix',y) = fx,'Y{x,y) for almost every y G R^. 

(F.IV) The class of functions J-q is bounded. 

{F.V) The class of functions Aiq := {m^/fx, ^ € J^q} is relatively compact with 
respect to the sup-norm topology on 

Under {F.I-2-IV), the conditional variance a^{x) of ^(Y) given X = £c is defined, for 
^' e ^g, by 

alix) = Var(*(Y) \ X = x) = / (^'(y) - m*(a;))^fx,Y(a;,y)dy. (5.2) 

Introduce the kernel estimator of m$(cc) defined on I, for any h £ [h'^, h'^], by 

n 

mq,,n,hix) := ^*(Yj)ro„,/i,i(a;), 

i=l 
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along with the following centering term 

IEm^;„(^;/i)=E{^(Y)K(^^)}/E{K(^^)}. 

5.1.1 Results 

We have now all the ingredients to state the result corresponding to the uncensored 
case. As mentioned above, Theorem 15.11 below is a complement of the successive results 
of Einmahl and Mason [12], Deheuvels and Mason 0] and Einmahl and Mason [13i] . 

Theorem 5.1. Let {h'^}n>i and {h'^}n>i be two sequences of positive constants fulfilling 
the conditions {H.l-2-3), with < h'^ < h'^ < I. Under the hypotheses {F.I-2-IV-V) 
and (i^.1-2-3), we have with probability one, 



{fill ^ 1/2 f ^ 'I 

— — , } sup sup±e„(a;)<^ m*;„(£c; h) - Em*:„(a;; h) \ 
"tL"n,"nj 21og(l//i'*) J v&e.^, tee/ ' J 

1/2 



„ 0(33) SUPvT/pT- 0-%{x) 1 

K^{t)dt sup ^ ' "^^.^7 ^ ' \ . (5.3) 



In Section 15.51 Theorem 15.11 will be shown to be a direct consequence of the technical 
result captured in Theorem 15.21 in Section 15.31 In the following Section 15.21 we first 
show that Theorem 13.11 naturally follows from Theorem 15.11 

5.2 Proof of Theorem [XT] 

In Proposition 15.11 below, we first establish the version of Theorem 13.11 corresponding 
to the case where G is known (i.e. with m^^f^ replaced by fh^^n.h)- To complete the 
proof of Theorem 13. II in the general case, the consistency of the Kaplan-Meier estimate 
will be helpful (see Lemmas 15.11 and 15.21 below). 

Proposition 5.1. Let {/i'„}n>i and {/i^}n>i be two sequences of positive constants 
fulfilling {H.l-2-2,), with < < < < 1. Under the hypotheses (A), (J), (F.1-2-3-4- 
5), {K.l-2-2>) and (0.1), we have almost surely 

lim sup sup sup ■ 



he[h'„,h'^]-4>eJ^x£i ^/2log{l/h<^) 

1/2 

l1 { 

V'S.F JRd x&I JX[X) 



snp{ / K\t)dt sup \ , 7 \ . (5.4) 



Proof. Recalling the definition ()2.2p of this function is uniformly bounded under 
(A) and (i^.4), for all ip ^ J-. This property, when combined with the VC property of 
J-, ensures that the class of functions J-^ := {^^^tj: £ J^} has a polynomial uniform 
covering number. Similarly, it can be shown that JP"<j, is a pointwise measurable class of 
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functions. Moreover, by (F.IV) and ([23]), the class M' = {E(^>^(y, C)|X = e 
is almost surely relatively compact with respect to the sup-norm topology. Therefore, 
we can apply Theorem 15.11 with Y = {Y,C) and ^ = <I>^. In this setting, note that 
(F.I) follows from (X), (-F.1) and {F.3). Now, observing that under the assumption (T), 
fori = 1,2, 



\\G(Z)J / \{G{Y)y-' 



X 



G{Z) 

the result of Proposition 15.11 is straightforward. □ 

Lemma |5. II below enables to complete to proof of Theorem 13. II in the case where (A)(z) 
holds. 

Lemma 5.1. Assume the hypotheses of Theorem \3. 1\ hold with (A)(i). Then, we have, 
with probability one. 



|-*(1) ^ ^ -vc / M f /log(l//i„)\ 

sup supsup|m,'^\(a^) -m^ ^(a;)| = o W — \,asn^oo. (5.5) 

Proof. Keep in mind the definition 12.41 of the functions Wn,h,i- Setting Wnj = 
^^Vhi^[h'„,h'^]^^Vx(^iYli=i \'^n,h,i{x)\i it is casy to check that Wn,i < oo. Moreover, ob- 
serve that, under (A)(i), 

i^vc(i) . . . ^ supi<^Q{sup^g^V(i) X - 
sup sup |m^y^(a;) - f^{x)\ < w^j = . 



Since tq < Th < 7g, the law of the logarithm for established in 15|] ensures that 



\ — - — I almost surely as n ^ oo, 
\ n J 

which allows to conclude to (j5.5p under {H.3). □ 

Lemma [5. 21 below enables to complete to proof of Theorem 13. II in the case where (A)(ii) 
holds. 

Lemma 5.2. Assume the hypotheses of Theorem \3.1\ hold with (A)(ii). Then, we have, 
with probability one. 



|-*(1) , X . ^| f /log(l//in)\ 

sup sup sup \m' '{x) - f^{x}\ = o[ J , as n ^ oo. (5.6) 
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Proof. Observe that, under (A)(m), 

sup sup \ml^n,h(.^) - f^'^,n,hix)\ < ^n,I SUp ^^J^^f^f^}.^ X SUp |G;(t) - G{t)\, 

^&:f he[h',^,h'J,] t<z„ t.n{i)Lr(t) t<TH 

where Wnj is defined as in the proof of Lemma |5. II and Zn := maxjZj : Si = 1,1 < i < 
n}. Clearly, min(G(Zn), (Z^)) > for all n > 1. 

Suppose that < p < 1/2 in {A){ii). Then, Theorem 2.1 in Q ensures that 

sup — G{t)\ = o(n~^) almost surely as n ^ oo, 

t<T„ 

which allows to conclude to ()5.6p under (H. 1-2-3) and (A)(m)(c). 

The proof in the case where (A)(zi) holds with p = 1/2 follows from the same lines, 
replacing the result of Chen and Lo [T]] by that of Gu and Lai 1^7']. Details are omitted. □ 



5.3 A useful technical result 

In Section [5.51 (see, especially. Lemma l5.5p . we will show that Theorem 15.11 is a direct 
consequence of a general empirical process result, captured in Theorem 15.21 below . This 
latter result describes the oscillations of a version of the multivariate empirical process 
indexed by an appropriate class of functions. This process is defined in ()5.7[) below. For 
any function "if £ J^g and any pair of continuous functions, Cij- and diji, defined on the 
compact / C TBJ^, set, for all a; G / and any h > 0, 



n 



h 



-nE{ (c^{x)^{Y) + d*(a;))i^ (^^) }. (5.7) 

Further denote by an the empirical process based on the observations (Xi, Yi),. . . , 
(X„, Y„), and indexed by a class of functions Q. Namely, for g £ Q, an{g) is defined 

by 

1 " 

an{g) = ^y^{g{Xi,Yi)-I^g{Xi,Yi)). (5.8) 

It is noteworthy that, setting for ^ £ J-'q, x £ I and h > 0, 

a;-u\ „ „^ 



??a=,h,*Kv) = (^c^(aj)^(v) + d,t(a;)ji^(^— ^j, for u, v G R'* x H". (5.9) 
the following relation holds, 

Wn,hi^, ^) = n^^^aniVxA^)- (5.10) 
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In view of (jS.lOp . the introduction of the process a™ provides a suitable and general 
set-up to study various types of kernel estimators (especially the density and regression 
functions estimators). See 0], and [13] for details. 

For future use, consider the following class of functions, 

G' ■■= iVxiM,'!'! - Vx2,h2,^2 '■ a;i,a;2 G /, *i,^2 e -^g,/ii,/i2 > 0}. (5.11) 



By (i^.3), arguing as in pages 17 and 18 of [Iz], it can be shown that ^ is a pointwise 
measurable class of measurable functions admitting a bounded envelope function and a 
polynomial uniform covering number. 

In other respect, introduce the following classes of continuous functions defined on I", 
indexed hy e J^q, 

Tc ■■= {c^{x) : £ J^g} and J^d ■= {d^i, (x) : £ J^g} . 

We will assume that the classes J^c and J^d are relatively compact with respect to the 
sup- norm topology on which by the Arzela-Ascoli theorem is equivalent to these 
classes being uniformly equicontinuous and uniformly bounded on I". 

We can now state, in Theorem 15.21 below, the technical result that will be instrumental 
in the proof of Theorem 15.11 (see Section 15. 5p . 

Theorem 5.2. Let {h'j^}n>i (^nd {/i^}n,>i two sequences of positive constants fulfilling 
the hypotheses (/f.1-2-3), with < < /i" < 1. Under the hypotheses {F.I-2-IV-V) 
and (K.1-2-3), we have with probability one, 

lim sup sup -j^^^^=^^^^- = sup ^(yj, (5.12) 

"^°^/ieK,?i^]*6.:^g V2n/i'^log(l//i^) ^ar^ 



where 



cj2(^) = supE|(c^(a;)^(Y) + dvi>(a;))^|X = a;j/x(a;) / K'^{t)dt. (5.13) 
The proof of Theorem 15.21 is presented in the following Section [57 



5.4 Proof of Theorem 15.21 

We will mostly borrow the arguments of 12], [l^ and First, we will consider the case 
where J-'g is reduced to {^}, for a given real valued, measurable and uniformly bounded 
function ^ defined on IR"^ (in such a way that (F.IV) and {F.V) are automatically 
fulfilled). Next, in Section [5.4.21 we will show how to extend this primary result to a 
more general class J^q. 
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5.4.1 The case where jFg = {^} 



Assume that J-q = {^'}, where ^' is a given real valued, measurable and uniformly 
bounded function defined on H'^. Without loss of generality, we will assume that k = 1 
in {K.II)[i). Moreover, we will suppose that ^' is such that o"(^') > (with (t{'^) as in 
(I5.13|) ). In this paragraph, our aim is to establish the following result. 

Theorem 5.3. Let {h'^}n>i and {h'^}n>i be two sequences of positive constants fulfilling 
the hypotheses {H.1-2-3), with < /i^ < /i" < 1. Under the hypotheses {F.I-2), {K.\- 
2-3), we have almost surely 

lim I sup {2nh'^\og{l/h'^)\ ^'"^ sn-p±Wnh{x,'^)\ = a{^). (5.14) 

The proof of Theorem 15.31 will be split into a lower bound part and an upper bound 
part. 

Lower bound part In this part, we do not work uniformly over h E /i^]. Instead, 
the result is stated for h = h'^. It is straightforward that, whenever Proposition 15.21 
below holds, such is also the case when taking the supremum over [h'^, h'^]. 

Proposition 5.2. Under the assumptions of Theorem \5.3l we have with probability one, 
liminf |sup±H^„,;,;^(a;,^')/ J2n/i;,^log(l//i;,^)| > a{^). (5.15) 

Proof. This result is the multidimensional extension of Proposition 3 in [l^]. A close 
look into their proofs reveals that our "extension" follows from exactly the same lines. 
We omit the details of these book-keeping arguments for the sake of conciseness. □ 



Upper bound part Here we claim that, under the assumptions of Theorem 15.31 we 
have, with probability one, 

V5>0,limsup( sup '-^^^0E^^tM\<{l + 2e)a{^). (5.16) 
n^oo [h6K,ftj;] ^j2nh'^\og{l/h'^) J 

For any real function Lp defined on a set B, we use the notation \\^\\b = sup^.^^ I<^(3^)|5 
and in the particular case where B = IR™',m > 1, we will write \\'^\\b = Moreover, 
for any class Q of measurable functions, we will use the notation 

||n^''^a„|L := sup |n"'"/^a„(5()|. 

First note that since ^ is bounded, there exists a constant < M^, < oo such that 
ll^'ll < M^. 

Now, fix e > in ()5.16p and introduce some constants 7 > 0, 5 G (0,a/4), and 
A G (1, (1 + 27)^/"^), which will be expressed in function of e > latter on. For any 
integer /c > 0, set 

nfc:=L(l + 7)'j, (5.17) 
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where [uj < u < [u\ + 1 is the integer part of u. 

For each k > 1 and n such that n^^i < n < n/^, consider the interval 

Next, set 



Rk := 



log A 

and consider the following partitioning of [h'^^ , h'^^ ^ 



+ 1, (5.18) 



K,A-=K,^, and h'^^y.= \'h'^^ for / = 0, . . . , ii^ - 1. (5.19) 

For each integer < / < Rk, we now include I in a union of Ji non- overlapping 
hypercubes, with sides of length d~^/'^5h'^^ ^ . These hypercubes are denoted by 

rM,i-{^Mj+]0,'i~'/'5/i'„,^i]'^} for l<j<Ju (5.20) 

and satisfy the following relations, for any integers k > \ and I = 0, . . . ,Rk (keep in 
mind that < S < a/4 and /i^ < 1 for n > 1). 

Ji 

lc(j Tk,i,, C (5.21) 
i=i 

Since the construction (I5.20p implies that the ^k,i,j, ^ ^ j ^ Ji, do not overlap, we can 
deduce that there exists a constant C := C{6), depending only on (5 > and /, such 
that 

C 

Ji < for k>l and < ^ < Rk- (5.22) 

In the sequel, we set 

Nk := {uk-i + 1, . . . ,nk}, (5.23) 

if rik-i < rifc, and Nj. := if ?T-fc-i = nk- Observe that for any initial choice of 7 > 0, 
the set Nk is non-empty provided the integer k is large enough. 

Our proof begins with the following decomposition, with k > 1 large enough to ensure 
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that iVfc / 0. 

= P 

< P 

< P 
+P 



max sup 1^ > (1 + 2e)(T(w) 



SUPa^gj |an(^cc,/i,5')| 



max sup 

max sup 



> (l + 2e)cT(*) 
> (l + 2e)cT(^') 



max max max 



max max sup 



> (l + e)f7(^) 



(5.24) 



2n,/i;5log(l//i;fJ 



vWbg(T7F) 

> eo-(^') 



Pl.fc + P 



2,fc- 



In view of (j5.24p . our aim is now to prove that Pi^fc and P2,fc are widely summable in 
k in order to apply the Borel-Cantelli lemma. 

Evaluation of Pi^fc ■ partitioning. Keep in mind the definition (j5.9p of the functions 
rixX^- 7 > and, for every A; > 1, < Z < Rk, I < j < Ji, and all (u, v) G TR^ x R', 
define 

9k,l,j{u,^) := ??^^_,^^.,,,^^_^,vi,(u,v). (5.25) 
Further introduce, for every A; > 1, the class of functions defined on M!^ x M'?, 

Gk-={9k,i,j-0<l<RkA<j<Ji}- 



In view of (j5.9p and (j5.25p . observe that, for every < / < i?^, 1 < j < J/ and all 



\\9k,i,j\\ + < 2{||cvi,|| X 11*11 + ||d^||}||i^|| =: Ml, 

where H^'H < M,j, < cxd by assumption. 



(5.26) 



Proposition 5.3. Assume the conditions of Theorem 15.31 are satisfied. If (jJ.^Op is 

fulfilled with < S < a/A, then we have, almost surely, for all e > 0, 



k^oo' I o<i<R. J2n,h'J^ ,\og{l/h'JJ 



lim sup < max 

Q<i<r 
i<j< 



< {l + £)a{-^). 



(5.27) 



19 



Proof. From ()5.22p . we have 



P 



l,k 



< 



> — max P 



^<j<Ji 



1=0 "'nk,l 

E—r- max Pi,fc,ij. 
h' i<i<Ji 



2nkKt,\og{l/h'J^^, 



> + 



(5.28) 



Arguing as in the proof of Proposition 4.1 in (see especially the proof of their state- 
ment (4.11)), it can be shown that 



max max \ai(gk,i,j{X ,Y)) < a^{^){l + £)h'J^^i. 



0<l<Rk neNk 
l<j<Ji 



(5.29) 



Thus, recalling the bound (15.26P on gk,i,j, the maximal version of Bernstein's inequal- 
ity (see, e.g.. Lemma 2.2 in [11]) when applied to the variables Ui = gk^ij{Xi,Yi) - 
'Egk,ij{Xi,Yi), i = 1, . . . ,n, yields under {H.2), 

' d{l+e/2) 



Pi 



Moreover, in view of (jS.lSp we have < 1 + [log(/i"^_^//i'„^)]/ log(A). Therefore, 
combining (fOOj) with ([OHI) . we get 



(5.31) 



Now, observing that, under {H.3), V7 > 0, X^feM ^"fe-i < the proof of (15.27P is 
completed by making use of the Borel-Cantelli lemma. □ 

Evaluation de P2,A; evaluation of the oscillations. Set ei = e/2 and 



B. 



n,k,Lh 



It holds that 
F2,fe = IP 



max max sup 

"'=l"ni.,i'"ni.,i + ll 



l 2nkKt,log{l/h'JJ 
2nh'^ \og{l/h'^) 



1 



n,h' 



2nkh'J^ l log(l//i' 



> 2eia{^) 



< P 



+ P 



max max sup 

^ nj,,i' nf,,l + ll 



2nfc/i'^iog(i//i;f,) 



max max sup 

neNkO<i<Rk-i ccerfei, 

^<j<Jl h^W h'' 1 

L nu,V n7,,i-fl-l 



B. 



n,k,l,h 



2nuh'J^^,\og{l/h'^l,) 



I d 



--: P2,l,fc+IP2,2,fe. 



> eicj(V') 

> eicr(V') 

(5.32) 
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First consider P2,2,fci and note that, for every n £ Nk and all h £ [h'^^ h'^^ ^j^^ 



B. 



n,k,Lh 



< 



+ 



n 



n 



(5.33) 



Some algebra enable to state 



liminf inin Jni 8^^-1,1, h > + J ^)Di{v)a{^), 

where A2 and Di(i^) are the constants involved in Fact IA.II (see the Appendix). 
Now set, for < / < - 1 and 1 < j < J;, 



(5.34) 



TP 



2,2,k,l,j 



max 

neNk 



sup 

h&lh' ,,h' ,,-,1 



Bn,k,l,h 



2n^h'J^^,\og{l/h'JJ 



In view of (j5.32p . we have 



1=0 j=i 



(5.35) 



Keep in mind the definition (|5.1ip of G' and introduce the class of functions 



^k,i,j ■■= < ic.!,{x)+d^{x)^)K 



X 



h 



,xGTk,u,heK^^i,h',^^i^,] \ eg'. (5.36) 



It is easy to check that J-'k,i,j is a pointwise measurable class of bounded functions 
admitting a polynomial uniform covering number, for all < / < and 1 < j < Ji. 
Moreover, arguing as above (see (15.290 ). it can be shown that uniformly over h G 
WnkpK,,l+i] and X G Ffc^/j, 



Var 



{c^{x)+d^{x))^{Y))K 



X -X 



V h 



< 



(5.37) 



for k large enough. Further observe that \\g\\ < Mi for all g G ^k,l,j, with Mi as in 
(j5.26p . Therefore, we can apply Fact lA.ll with r = 2a{ip) and p = T^J^j A2. When 
combined with ()5.34p . this yields 



P 



2,2,fc,«,i 



< P 



max sup 



> 2 1 + 




Di{u)a{i;) 



< P 



max \\n 

l<n<nj. 



< 4^5- 



(5.38) 
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In view of ()5.35p and (15.380 . and arguing as in ()5.3ip . it follows that 



(5.39) 



Turning our attention to P2,i,fc) set, for < / < iife — 1 and 1 < j < Ji, 



F 



P 



max sup 



he\h' ,,h' ,,,] 

In view of (|5.32p . it follows that 



2n,h'J^^^log{l/h'/j 



> eicj(V') 



i?,fc-i Jt 

1=0 j=l 



(5.40) 



Next consider the classes of functions 



^k,i,j ■= ^9k,i,j - Va^,h,^,xe Tk,i,j,he K^,/,/infe,i+i]| C G', (5.41) 

with g^^ij defined as in (j5.25p . To evaluate ^2,i,k,i,j, our aim is to apply Fact lA.l] once 
again. First note that \\g\\ < Mi for all g S ^k,i,j (with Mi still as in (|5.26p ). Now we 
shall get a suitable upper-bound for sup Var(g(X, Y)). Set /3 := + 1 < oo 

and introduce, for any real valued function (j) defined on I" and all 6 > 0, 



LJ0(5) := sup||0(a;) - (f){y)\ : \\x - y\\ < 6, and x,y £ /"j. 



(5.42) 



Making use of the classical inequality (a + 6)^ < 2(a^ + 6^), it can be derived, from ()5.9p . 
dOnH and ([525]), that, for A; > 1, < / < iifc and 1 < j < J;, 



E 

< 2E 



{ ((c* (xk^i-j) -c^{x))^{Y) + {xk^ij) -d^{x)yK( ""^'"'^ ^ ^ ' 



2W.[{c^{x)^{Y)+d^{x)f{K{^^ 



Xk,i,j-X^ _^/a;-X\\2 



< 4/3{<«^,,,) V<«^.,,)}e{k 

+ 4/?|||cvi>|||a V ||dxt||?.}E|(^i^ 
=: (/) + (//). 



Xk,l,j - 

Xk,i,j-X\ rx-Xw^ 



(5.43) 
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Set 



Assuming as before that k = 1 in {K.II){i), we obtain 



(5.44) 



K 



<\\K\\H{XeTu,i,j]. 



So we have, for k large enough, and for every < / < — 1 and 1 < j < Ji, 

[I] < 4/3{<(<^^jv<(<^,,)}||i^f p(XGrfc,;,,o 

< ^(3{u;l{5h'^jyul{5h'^j} \\Kf \\fx\\i'^h'/^,i. (5.45) 



In order to bound (//), we wih employ the assumption {K.I). Let Bi and B2 be the 
two functions respectively defined on IR and [1, +00) by 



1/7<A<7 

The assumption {K.I) ensures that 



Bi{5) = sup / {K{x)- K{x + vi)) 

|u|<<5 jRd 

^2(7) = sup / {K{\x)- K{x)fdx 

7<A<7 J'R'i 



lim Bi{5) = and lim BiiX) = 0. 

5^0 A^l 



(5.46) 
(5.47) 

(5.48) 



Let A := j^,{K{{x -t) /h) - K{{xk,i,j -t) /h'^J)^dt. Setting u = {xk,i,j -t) /h'^^ ^, we 
obtain, from the definitions (j5.19p . (j5.20p . ()5.46p and (|5.47p . and the Cauchy-Schwarz 
inequality. 



+ 2 



K\ ; ^ + U— ^ 

n n 
K{^^^ + u%L 



K{u) 



du 



Km 



/ 25 



h 

K\ u 



dvL + 



Rrf 



Ki u 



KivL 



K{u) 



K{u) 



du 



+ B2{X) + 2. 2Bi 



/26_ 



B2{X). 



Therefore, we have A < ( ,^/2Bi{-^) + 



' 25 \ 



^2(A)^ , in such a way that 



(//) < 4{||c^||?. V ||dv,||?.}/3||/x||/<^(j2Si(2V\^) + 
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From (I5:i3|) and (lOSD . setting B = 4||/x||/-/3{ + {\\c^\\% V \\d<f,\\%)\K\l'^ , it 
holds that 

Var(5fc,/,,(X,Y)-7?,,;,,v,(X,Y)) < B[iu;^,i5h'^J V u;ji5h'^J 



Now set for A: > 1, < Z < i?fc - 1 and 1 < j < J/, 

ali^^i^) = sup {Var(5(X, Y)) : g G 

By selecting 5 > small enough and A > 1 close enough to 1, the continuity of the 
functions and d-i, when combined with (j5.48p implies that, for k large enough, 

where ^2 and are the constants involved in Fact lA.ll Moreover, arguing as 

above, it is easy to check that J^^ i j C ^' is a pointwise measurable class of bounded 
functions with a polynomial uniform covering number for every < / < ii^ — 1 and 
1 < j < Jj. The refore, we can apply Fact |XT] with r = ei(T(V')/[(l + ^j2/A2)Di{u)] 
and p = T\j2lA2^ which yields 



max ||nV2«„|| > m{y){T + p)J2n^h!^l, log(l A^,",;) 



(5.50) 



Arguing as in (j5.3ip . it follows from (j5.40p and (|5.50p that 



By combining (j5.39p and (j5.5ip we conclude under {H.?,) that X]a:>i -"^2,^ < oo, which 
achieves the evaluation of the oscillations part. The statement X]fc>i IPfc < oo now 
directly follows from ()5.3ip , completing the proof of (I5.16P . 

By combining (|5.16p with Proposition 15.21 the statement of Theorem 15.31 is straightfor- 
ward. 



5.4.2 The general case 

To extend the result of Theorem 15.31 to the general case, only the "upper bound" part 
has to be extended. From the preceding paragraph, it is straightforward that for any 
finite subclass Qq <Z J-q, we have, with probability one, 

sup^gg sup^g^|W„,h(a;,1')| 
iimsup sup < sup (7(yj. (5.52j 

n-*oo h(^[h'^,h'^] ^/2nh'^\og{l/h<^) *eg. 
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Here we shall show that (I5.52p can be extended to the entire class Tq. The following 
couple of Lemmas are directed towards this aim. Set 

Ct^ := sup{||c*||/<^ : ^' G ^g} and D:f, ■= sup{||(ivi>||/<. : ^ £ J^J. 

Keep in mind that, by {F.IV), the class J-q has a uniformly bounded envelop function 
T, with T(y) > sup^g^^ ^(y), y G R^. 

Lemma 5.3. For all e > 0, there exists a finite subclass Qq^e C J'q such that, for all 
^ £ J-q, for n large enough, 



min sup 7-7 E 



X-x 



< e. 



(5.53) 



Proof. Let J be a compact of R'. For </> G J^q, x e I and h G [h'„, h'^], we have 



{c^{x)^{Y) + d^{x) - c^{x)^{Y) - d^{x)YK' 



X-x 



{cvi,(a;)^'(Y) + d^{x) - c^{x)(t){Y) - d^{x)Yli^\y^_^\<h/2} 



< j^n{c^j(y)+D^f'i{Y^j,\^--.\<h/2}]\\K'\\oo 

+ r/(J) J {c^{x)^{Y) + d^{x) - c^{x)<P{Y) - d4x)Ydy, 
=■■ W + W- 
where r/(J) := sup(3._y)gjc«x j /x,Y(a;, y)||-?^^||oo- 

To evaluate the term (/), first observe that the Holder inequality gives us, for all s > 1, 



(/) < ^K^\\ooa^/'\\fx\\% supP(Y ^J\X = x 



where a := sup^.g/ E[(Cjr^T(Y) + Djr^)'*|X = a;] < 00. Under our continuity assump- 
tions, and from Scheffe's lemma, the function x — > P(Y G -jX = a;) is continuous from / 
to the space of all probability measures on Borelian sets of IR'? endowed with the topology 
of weak convergence. Thus, the set {P(Y € -jX = a;), a; G /} is compact in this space, 
which implies, in view of Prohorov's theorem, that it is uniformly tight. Consequently, 
for all e > 0, there exists a compact C H'^ such that P(Y G Je|X = a;) > 1 — e/2. 
Finally, we obtain that there exists a compact J = such that, uniformly in a; G /, 

To evaluate the term {II), we will use the fact that J-q is a VC subgraph class of 
functions, which ensure that it is totally bounded with respect to dn^ , where Qs is the 
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uniform law over . We can deduce that for all 5 > 0, there exists a finite subclass 
Qq^i d T for which 



sup min / (^'(y) - '/'(y))^dy < b. 



Moreover, the relative compactness of the classes Tc and J-t> implies the existence of 
finite subclasses ^5,2,^5,3 C Tq such that 

sup min ||c>i, — c^H/ V sup min ^d^n — d^\i < 8. 

Combining the facts that 

sup llc^rll/a < 00 and sup / \I'(y)^(iy < 00, 

and choosing 5 > small enough, we have 

sup min sup / {c^,(a;)^'(Y) + d^,(a;) - c^^{x)(pi(Y) - d^.^{x)]^dy < £/{4r]{Je)), 
^&Tq 'Pi,4>2,4>3 xei Jj^ 

where the minimum is taken over Qg^i x Qg2 x Qq^s- For any triplet {<l)i,(j)2,4>3) G 
Gq,i X Gq,2 X Gq,3 for which there exists (p G J^q such that 

sup / {c^{x)(t){Y) + d^{x) - c^^{x)(t)i{Y) - d^,^{x)Ydy < e/{4r]{Js)), 

we select one of these </) G to construct the subclass Qe- Applying the triangle 
inequality, we obtain 

m < f , 

which completes the proof of the lemma. □ 

Set e > and tiq such that (|5.53p holds for all n > uq. For any "^jCp £ J^q, we define, 

:= sup sup h supE {c*(a;)^(Y) + (i^(a;) 

n>no h&[h'„,h'^] xel L 

-c4x)cP{Y)-d4x)}'K'[^^)' . 

Lemma 5.4. Under the assumptions of Theorem \5.S\ there exists an absolute constant 
A such that, for all e > 0, we have almost surely, 

hmsup sup ^ ^= = < ADi{vye. (5.54) 

n^oo /ig[/i^,h^] Y^nn'*log(l/ft'*) 
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Proof. To establish Lemma l5.4| we intend to apply Fact lA.ll Consider the classes of 
functions 

jr(e, h) := {r]x,h,^ - Vx,h,^ : d^i"^, (f)) <e,x e /}, for/i G /i"], 

where the functions r]x,h,^ are defined as in (j5.9p . For any integer k > 1, set nfc = 2^. 
Further setting 

SUPd2(^M<eSnVxeI \Wn,h{x,'i>) - Wn,h{x,(l))\ 

Qk '■= max sup 



nk-i<n<nk he[h'„,h'^] ^/2nh<^ log{l/h<^) 

we have 

Qk = max sup 



nfc-i<"<nfe/ie[/i;,h;;] y^2nh'^ log(l//i^) 

Now, in the same way as above, consider the following partitioning of the interval 

Kfe'Kfc-J ^ IK^K], for nfc_i <n<nk, 

ii ill 

h'^^y.=\'K^, l = O...Rk-l, (5.55) 
where satisfy the condition 



Rk 



+ 1. (5.56) 



log(A) 

For any < I < Rk, introduce the classes of functions 

^k,ii^) ■■= {V<r,h,^ - Vx,h,4> ■ d^i^A) <e,xel,he [hn^^j,K^ i^i}- (5.57) 

In view of this definition of J-k,i{s), we have, for any function v £ J'k,i{^) and for k large 
enough, 

/i''^Var(t;(X, Y)) < d^i"^, 0). (5.58) 
Thus, from ()5.57p and (j5.58p . setting i ^ '■= sup{Var(t;(X, Y)) : v G J^k,i{£)}, we have 

Furthermore, for any function v G J'k,i{£), we have ||f|| < Mi (where Mi is defined 
as in (|5.26p ) and we can show that each subclass J-k,i{£) is pointwise measurable and 
such that J^k,i{£) C Q, where ^ is a pointwise measurable class of functions admitting 
a bounded envelope function and a polynomial uniform covering number (to do so, we 
use the same arguments as those used to establish that Q' had thisproperty along with 
the fact that !Fq is VC] we refer to the proof of the Lemma 5 in [12] for more details). 
So, the assumptions of Fact lA.ll are fulfilled, with r = eV^A^ and p = t\J\JA^. By 
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applying this result, we have, with Nk = {rik-i + 1, ■■■,nk} as usual, and applying the 
same techniques as those used to establish (j5.38p . 



P 



£,k,l 



P 



Thus, 
P 



e,k 



,1/2 



n ' War 



max 

l<n<7ife 



2^fc<.,/log(l/<,/) 



> V\'ii{l + ^/T/A2)Di{iy)e 



P 



,1/2 



n ' \\a 



max max 



Rk-i 



< 



-h 



rrd 



which implies the lemma [5^ via Borel-Cantelli's lemma. □ 



Combining the lemmas 15.31 and 15.41 with (|5.52|) , we obtain the uniformity over the class 
J-'q, which ends the proof of theorem 15.21 in the general case. 

5.5 Proof of Theorem 15.11 

In this part, we show how the results obtained for the process Wn^hi^: ^) can be trans- 
posed to nonparametric functional estimators of the regression, that is how Theorem 
15.11 directly follows from Theorem I5.2[ Towards this aim, first introduce the following 
quantities 



fx,n,h{x 

Jx,h{x 



X 



/ ^(y)/x,Y(aj,y)dy; 
1 " 



X — X, 



1 " 

i=l 



X- Xj 

h 



1 



^r^,n,h{x)=^E\-^^|JiY)K^ ^ 



)}• 

x-X 



)}■ 



Now, choosing c*(a;) = l/fx{x) and ci*(a;) = — mii>(a;)//x(a;) in the definition ()5.7p 
of Wn^hix, it is easy to show that, for all /i > the following relation holds 



Wn,h{x,^)=nh 



df rii,,n,hix) _ r^^hjx) _ rq,{x){fx,n,hix) - fx,h{^)} 
I fxix) fx{x) flix) 



}. (5.59) 
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We can now state Lemma l5.5( according to which Theorem 15.11 is a direct consequence 
of Theorem I 



Lemma 5.5. Under the assumptions of theorem \5.S\ we have, almost surely, 

1 



F ■- 



sup sup sup ■ 



haW^^'A] *eJ^, x<^i yjnhd- log(l//i'^) 



oil). 



fX;n{^,h)J 

Proof. The proof is identical to that of Lemma 10 in [l2] (see also Lemma 4.7 in Q]] 
and is omitted for the sake of conciseness. □ 



5.6 Proofs of corollaries 14. IL 14.21 and 14.31 

Corollary 14. II being a direct consequence of Theorem 13.11 with !F = {]I[o,t] , i < ''"o < ^i?}; 
details of its proof are omitted. 

5.6.1 Proof of Corollary [472] 

Using the following notation 

E{v„....,.„g(=q^)} 

we will first show that 

lim sup sup supsup |/^^^(t;a;) - //j^f(t;a3)l = 0. (5.61) 

Since ^ ^ 0, and t'^ > ^^'"felV^"^ we have for any s > 
lim sup sup sup sup \ f* ^{t; x) - fh,i{t; x)\ 

± Qn{x) SUpt<^^ SUPo<^<s ~ ^)} 

< lim sup sup __ — . 

"^°°heK,ha»=e/ V21og(l/^'^) 

Set s £ iO,TH - To). By applying Theorem O with J^^ = {'^[t-e/2,t+e/2], t < tq, < 
£ < s}, we have with probability one 

Vnf? ± Qnix) SUPi<^(, SUPo<f<s ^1 fn^hA^'^ ^) " fhA^'^ ^) } 

lim sup sup = 

= I / KHu)Cu sup ^''^'7f-^--*'-' |"'. (5.62) 
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But, by (A), (F.2) and (D), we have 



limsup sup cr^{x) = 0, 



and then the right-hand term of ()5.62p can be rendered arbitrary small, which enables 
to conclude to (I5.6ip . 

As for the deterministic term, first observe that 



lim sup sup sup sup \ fh e{t; x) — f{t; x) > 



lim sup sup sup sup 

helh',^,h'j^] ^6K/^] 3=6/ t<To 



(7i^{Ve[t-|,+|]}^ 



x-X 



}-fx,Yi^,t))fx{x) 



+ 



/xM-iE{/f(21^)})Mv-(x.4. 



By {F.2) and (D) it is enough to establish the two following statements to conclude the 
proof of Corollary 14.21 

x - 



lim sup sup sup |-— T-E 

he[h'„,h'J^] x&i t<To ^ h'^f^ 



and 



lim sup sup sup I fx{x) — 7-7E 



K 



h 



x-X 

h 



/x,y(a;,t)} =0 (5.63) 



But, for any /i > 0, ^ > 0, cc G /, t < tq and n > 1 we have 



1 



t+i 



i h-L 



{fxyix - hv,y) - fx,Y{x - hv,t))dyK {v)dv 



and 



+ / [fxxi^ - hv,t) - fx,Yi^,t))K{v)dv 
fx{x) - ■1e{k(^^) } = {fx{x - hv) - fx{x))K{v)dv. 



(5.64) 



In view of these two last results, and since /i'^ and decrease to as n ^ 00, it is easy 
to see that (j5.63p and (j5.64p are direct consequences of {K.2) and {D). 
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5.6.2 Proof of corollary 1431 

First observe that 

lim sup sup sup sup IX^ h ^(^5 ^) ~ -^(^S ^) f 

1 

= lim sup sup sup sup 



' h&[h'M t<ro (1 - F{t; x)){l- ^) 

{ iflh/t' ^) - fit' ^)) (1 - ^(*; ^)) + fit' ^) (^nV(i; ^) - F{t; ^: 

In view of Corollaries 14. II and 14.21 and since F{tq) < 1 and f(t; x) is bounded for t < tq 
and 03 G I, it is enough to prove that 



lim sup sup sup (Ffi{t;x) — F{t;x)\ = 0. 



' h(^[h'„,h'^] X&I t<TO 

But, for any /i > 0, € /, t < tq and n > 1 we have 



Fh{t-x)-F{t-x) 



1 



fxix)^ Jy<t 



I / / K{v){fx,Y{x - hv,y) 



-fx,Yix;y))dvdy x /^(aj) - ^ ^ /x,y (a^, (^E|/i:(^^-^) | - /^(a^))}- 

By (-F.2), (D) and (|5.64p . it is enough to evaluate the quantity J^^^ Jj^^ K{v)[fx,Yi^~ 
hv, y) — fxyix; y))dvdy. Towards this aim, set e > 0. Since / is compact and Tp < oo, 
the assumption (D) implies the existence of a constant t{e) such that Va; G 

/ fx,Y{x,y)dy 
Jv<t(e) ^ 



So, by {K.2) and (D) and by imposing h < min(-^ 



2 \ vK{v)\dvBa{To-t(e)) 



we have 



/ / K{v){fxyix - hv,y) - fx,Yix;y))dvdy 

Jy<t JR'' 

: / / [fxyix - hv,y) - fxyix]y))dy 

Jmfi I Jy<t{e) 

I ifx,Yix - hv,y) - fx,Yix;y))dy \dv 

Jy(LW)-A ' J 



+ 
< e 



which concludes the proof of Corollarv 14.31 □ 
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A Appendix 

The following fact has been stated in [2^ (see also [2^ for a similar result). 



Fact A.l. Let T he a pointwise separable class of functions satisfying 



supVar(/(Y)) < r^h, 



with T,h > 0. Assume there exist M, C, > fulfilling, for all < e < 1, 



sup |/(y)| < M. 



Choose p > arbitrarily. Then, there exist a universal constant ^2 > and a parameter 
Di{v) > depending only upon v such that, if h > satisfies 



then, setting T.n{g) = YJj=i {dO^j) - ^(^(Y))} for g e J^, we have 

f( sup \\Tmi-)\\T > {t + p)Diy/nhlogil/h)) <4exp(-.42(-)'log(l//i)) . 

\l<m<n J \ T / 
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